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Abstract 


We studied relationships among background characteristics, behavioral infractions, 
punishments, attendance, and educational achievement, using longitudinal data of students in 
grades 1-6 and 8-12. We estimated how much hypothesized early improvements in 
educational achievement or sustained improvements in behavior and attendance might ultimately 
increase educational achievement in grades 6 and 12. We also calculated similar estimates for 
increasing the rate of promotion from grade 1 to grade 2. 

Reducing infractions and punishments to zero and days absent to the average observed 
level in grades 1-6 would increase the percentage of students attaining proficiency on sixth- 
grade achievement tests by approximately 3% in literacy and 4% in mathematics. For purposes 
of comparison, increasing first-grade educational achievement by 0.1 SD would increase 
proficiency rates in grade 6 by approximately 2% in either area. 

For students in grades 8-12, the outcome variable was attainment of all four ACT 
College Readiness Benchmarks, which are indicators of students’ readiness to take typical first- 
year college courses. Reducing infractions and punishments to zero and days absent to average 
levels would increase attainment of the Benchmarks by about 2%. Increasing eighth-grade 
achievement by 0.1 SD would increase attainment of the Benchmarks by approximately 3%. 

As measured by average changes in scale scores, our results indicate that the benefits of 
improved prior achievement substantially fade with time. For example, a 0.1 SD increase in 
grade 1 literacy score corresponds to an expected increase of less than 0.04 SD in grade 6 
literacy score. To endure over time, therefore, the benefits of improved prior achievement must 
be enhanced by sustained interventions (in this study, on behavior and attendance). 
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Exploratory Analyses of the Long-Term Effects of Improving Behavior, Attendance, and 
Educational Achievement in Grades 1-6 and 8-12^ 

The educational achievement of many students in the U.S. is short of what they are 
capable of, and is insufficient for the well-paying jobs that will likely be available in the future. 
Furthermore, underachievement continues to be concentrated among traditionally underserved 
demographic groups defined by race/ethnicity and family income, which in turn undermines 
social stability and promotes the continued underachievement of future generations. Although 
the dimensions and consequences of this problem are understood, there is little agreement on its 
causes or on ways to fix it. 

Many different recommendations have been made for improving students’ educational 
achievement. They include restructuring public education bureaucracies, promoting charter 
schools and voucher systems, imposing accountability systems on schools and teachers, 
improving teacher training, raising requirements for licensure, instituting special incentives for 
excellent teaching, improving curricula and methods of instruction, encouraging greater 
involvement of parents, instituting special programs and services for at-risk students, raising 
requirements for graduation, adopting stricter policies on student behavior, and raising students’ 
and parents’ awareness about the importance of education, among others. Acrimonious debate 
accompanies many of these suggestions, particularly those that involve punitive sanctions, 
spending large sums of money, or shifts in political power. Moreover, when particular remedies 
are implemented, there typically is only modest improvement, as well as inconsistencies in the 
results among jurisdictions and research studies. 


* This study is based on data maintained by the Arkansas Department of Education, and is published with its 
permission. 
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There are numerous reasons why improving edueational aehievement on a large seale is 
diffieult. One fundamental problem is that although many proposed drivers of aehievement are 
plausible, we do not know in great detail how they relate either to aehievement or to eaeh other. 
Knowing these relationships would require assembling longitudinal data on all the potentially 
relevant variables over the entire span of students’ sehooling, then modeling all of the eomplex 
relationships. It is not praetioally feasible to obtain and organize sueh data for a large sample 
that is representative of some population of interest. Furthermore, even if a eomprehensive 
longitudinal data set could be constructed, analyzing the data would be very complicated. 
Research on educational achievement therefore proceeds with incomplete sets of variables over 
limited periods of time on samples of students that might not be representative of larger groups. 
Prior Achievement, Behavior, and Attendance 

We know that among the potential determinants of current educational achievement, prior 
achievement is very important. Indeed, prior achievement is an essential component of value- 
added models of school and teacher effectiveness (e.g., Sanders (1998); Nye, Konstantopoulos, 
and Hedges (2004)). Sawyer (2008) found that students’ performance on the eighth-grade test 
EXPLORE (ACT, 2011a) was more important in predicting their performance on the ACT 
(grades 11/12) than their background characteristics, the high schools they attended, the courses 
they took in high school, and the grades they earned in their courses. Moreover, prior 
achievement interacts with the other variables: Students with higher EXPLORE scores 

benefitted more from taking more rigorous courses and earning higher grades in those courses 
than did students with lower EXPLORE scores. 

There is also evidence relating behavior and attendance to educational achievement. 
Using data from the NELS:88 survey, Kauffman and Bradbury (1992) found that eighth-grade 
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students who reported that they had “been sent to the offiee” more than twice for misbehavior 
were more likely to have below-basic scores in reading and mathematics. The same result 
pertained to students who reported missing school five or more days in the preceding month, as 
compared to those who reported missing no days. The results in this study were adjusted for 
socioeconomic status (SES), race/ethnicity, and gender, but not for prior achievement. 

In another study of middle school students, Lassen, Steele, and Sailor (2006) examined 
the relationship between suspensions for misbehavior and subsequent scores on reading and 
mathematics tests. They found standardized regression weights of approximately -0.1 1 in either 
subject. Their regression models did not include background characteristics or prior 
achievement as covariates. 

Jennings and DiPrete (2010) studied the relationship between teachers’ ratings of student 
behavior and the students reading and mathematics scores in grades 1-3. They reported 
standardized weights of 0.04 to 0.14. The relationship between behavior and reading 
achievement increased with grade level, but the relationship between behavior and mathematics 
achievement decreased with grade level. 

Analyzing data of elementary and middle school students in Philadelphia, Gottfried 
(2010, 2011) found statistically significant relationships between attendance and standardized 
test scores in reading and mathematics, after controlling for background characteristics, prior test 
scores, teacher characteristics, and neighborhood characteristics. Effect sizes related to the 
attendance variables were 0.05 - 0.10. Gottfried (2009) also found that unexcused absences, but 
not excused absences, have a negative relationship to test scores. Chang and Romero (2008) and 
Ready (2010) found that good attendance is especially important for socioeconomically 
disadvantaged children. 
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There is a larger literature showing that misbehavior and/or poor attendanee prediet 
dropping out of high school (see, for example, Rumberger (1995); Rumberger & Larson (1998); 
Allensworth & Easton (2007); and Balfanz, Herzog, & Mac Iver (2007)). Finn, Fish, and Scott 
(2008), analyzing the NEFS:88 data, found that well-behaved high school students are also more 
likely than moderately misbehaving students to participate in postsecondary education. 
Moderately misbehaving students are not more likely than seriously misbehaving students to 
participate in postsecondary education, however. 

The present study is based on data maintained by the Arkansas Department of Education. 
The data pertain to students who enrolled in Arkansas public schools in academic years 2004- 
2005 through 2009-2010, and are supplemented by the PLAN and ACT scores^ of the students 
who were enrolled in grades 10-12 during this time. The data include four important types of 
variables: background characteristics, behavior (infractions and punishments), attendance, and 
educational achievement (as measured by test scores). 

The research questions addressed by this study are: 

• What are the principal predictors of promotion from grade 1 to grade 2? 

• How much could consistent improvement in behavior and attendance potentially 
improve educational achievement in the long term? 

• How much could early improvement in educational achievement potentially improve 
educational achievement in the long term? 

• What are the combined benefits of consistent improvement in behavior and 
attendance and early improvement in educational achievement? 


^ The ACT (ACT, 2007) measures students’ knowledge and skills in written English, mathematics, reading, and 
science. PLAN (ACT, 2011b) measures knowledge and skills in the same areas, but at the tenth-grade level. 
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The analyses in this study are exploratory: They are intended to diseover relationships, 
rather than to confirm a formal theory. Furthermore, as will be discussed below, variables such 
as neighborhood characteristics, teacher characteristics, and student grades were not available to 
be included in the analyses. A few school-level characteristics (e.g., number of students) were 
available, but we chose to model school characteristics in this exploratory study only as random 
effects of the model intercepts. Finally, although the data are longitudinal, they come from a 
single state. Nevertheless, we hope that the results will provide useful preliminary answers to 
the research questions posed and will assist in the further development of theory. 

The outcome variables in our study primarily involve students’ educational achievement, 
as measured by test scores. The data available could also permit modeling students’ progression 
in grades over time, dropping out, and timely graduation. In this study, we modeled promotion 
to grade 2, test scores, and proficiency status as outcomes. Modeling other outcomes would be 
worthy goals of future studies. 

Data 

The target population for this study consists of students who enrolled in Arkansas public 
schools in the academic year ending in June 2005 (denoted as AY2005). We studied two 
separate subsets of these students: those who enrolled in grade 1 in AY2005, and those who 
enrolled in grade 8 in AY2005.^ Data for the AY2005-gl students continued through AY2010 
(when they would typically have been enrolled in grade 6). Data for the AY2005-g8 students 
continued through AY2009 (when they typically would have been enrolled in grade 12). 


^ The time span for which data were available did not permit including the data of students in grade 7 in the 
analyses. 
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Variables 

Table A-1 in the appendix deseribes the variables for these two eohorts. The baekground 
variables include gender, race/ethnicity (categorized as membership in a minority group), 
English language spoken at home, and income-related variables (e.g., eligibility for free or 
reduced lunch). The attendance variables for each student are the numbers of days absent, by 
grade level. 

The Arkansas Department of Education has a detailed coding scheme of behavioral 
infractions and punishments. Eor modeling promotion to grade 2, we used the type and the 
frequencies of infractions, and the type and the frequencies of punishments, as predictors. To 
simplify the modeling of the other outcome variables, we used the total number of infractions 
(summed over all types) and the total number of punishments (summed over all types) as 
predictors. 

Arkansas requires that public school students take various state-sponsored tests, which 
are developed by private companies under contract to the state. Students in grades 1 and 2 took 
norm-referenced tests in literacy and writing. Students in grades 3-6 and 8 took criterion- 
referenced tests in literacy and mathematics."^ Scores and proficiency levels on the grade 6 
literacy and mathematics test served as the final outcome variables for the AY2005-gl cohort. 

Students in grade 9 took norm-referenced tests in literacy, mathematics, and writing. 
Students in grades 10-12 took various end-of-course exams; among these, we used only the 
score in grade 1 1 English/Eanguage Arts in our models. We used the PLAN Composite score 
and the ACT Composite scores as measures of students’ educational achievement in grades 10 
and 1 1/12, respectively, instead of scores on state-sponsored tests. 


Students in grade 5 also took a criterion-referenced test in science, but we did not use it in the models. 
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Our models do not contain other variables that have been hypothesized or shown to 
predict student achievement. There are many such variables. Examples include: 

• psychosocial characteristics, such as engagement, academic discipline, and 
socialization skills (Akey, 2006; Jones & Byrnes, 2006; Collaborative for Academic, 
Social, and Emotional Learning, 2007; Casillas, Allen, Kuo, Pappas, Hanson, & 
Robbins, 2011; Demaray & Jenkins, 2011); 

• parental involvement (Ogbu, 2003; Charles, Roscigno, & Torres, 2007; Jeynes, 
2007); 

• teacher effects (Nye, Konstantopoulos, & Hedges, 2004); 

• curriculum (Edvantia, 2005; The Core Knowledge Eoundation, 2012); 

• school socioeconomic characteristics (Griffith, 1997; Konstantopoulos, 2006); and 

• students’ and parents’ beliefs and values about the importance of education (Dillon, 

2010 ). 

Note that psychosocial characteristics (first bullet) might predict the behavior variables 
(infractions and punishments) used in this study. Psychosocial characteristics might also, 
however, directly predict educational achievement as measured by test scores separately from the 
behavior variables. In other words, psychosocial characteristics might have both direct and 
indirect effects on educational achievement. 

As we noted previously, it is not feasible to collect data on all the variables that could 
contribute to a large group of students’ educational achievement. With observational data, 
omitting variables in a model can, in principle, bias the estimates of coefficients of variables 
included in the model. On the other hand, we have included in our models what we believe is 
the primary predictor of current achievement; namely, prior achievement. Acknowledging the 
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limitation of potential omitted variable bias, we hope that this study nevertheless will contribute 
to the development of strategies for improving students’ educational achievement. 

Description of Student Populations and Samples 

The students who enrolled in grade 1 or grade 8 in AY2005 are summarized in the “All 
students” columns of Table A-2 in the appendix. Approximately a third of the students in either 
population are minority. Both populations contain substantial proportions of economically 
disadvantaged students; for example, 51% of the AY2005-gl students and 42% of the AY2005- 
g8 students were eligible for free or reduced price lunch. 

All analyses in this study are based on subsets of these two populations, as determined by 
the schools in which they enrolled. About one-third of schools reported discipline data (behavior 
infractions and punishments) for the entire time span of the study. Because all of the models are 
based in part on the discipline variables, we estimated them only from data of students who 
attended schools that reported these variables. For example, of the 41,432 students who enrolled 
in grade 1 in AY2005, 18,769 enrolled in schools that reported discipline data. In Table A-2, 
this subset is designated the “AY2005-gl cohort analysis file”. 

There are three groups of models in this study (see Table 1 on page 11), as defined by 
final outcome variable: 

• Promotion to grade 2, 

• Educational achievement in grade 6 (as measured by scores on state achievement 
tests), and 

• Educational achievement in grades 1 1/12 (as measured by ACT scores). 

Each group of models is based on a different sample. 
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The models for predieting promotion to grade 2 are based on all students in the AY2005- 
gl cohort analysis file. The models for predicting educational achievement in grade 6 are based 
on the 14,420 students who were enrolled in grade 6 in AY2010. This subset is designated the 
“AY2005-gl at-grade-level analysis file”. Of the remaining 4,349 students, approximately 900 
were retained in grade 1 (and were focus of the models for predicting promotion to grade 2). 
Approximately 600 students were retained in grades 2-6. The remainder either transferred to a 
non-public school or moved out of state before grade 6. 

The analysis file for the AY2005-g8 cohort was defined by three conditions on students: 

• attended schools that reported discipline data, 

• enrolled in grade 12 in AY2009, and 

• took PLAN in grade 10 and the ACT in grades 1 1/12^. 

Among the 37,891 students who enrolled in grade 8 in AY2005, 10,196 met all three conditions. 
This subset is designated the “AY2005-g8 ACT-tested at-grade-level analysis file”. Note that 
because it pertains to students who remain enrolled in school through grade 12, this file is not 
representative of all students, some of whom dropped out of school. The outcome variables in 
the analyses of this file involve observed ACT scores, however: We do not attempt to make 
inferences about the ACT scores that students might have obtained if they had not dropped out. 

Students represented in either the AY2005-gl cohort analysis file or the AY2005-gl at- 
grade-level analysis file are slightly less likely to be minority, from non-English speaking homes, 
and eligible for free or reduced price lunch than are all students (Table A-2). The differences 
among the three groups are five percentage points or less. Students in the AY2005-gl at-grade- 
level analysis file also had slightly lower average numbers of infractions and punishments and 

^In the data available for this study, over 90% of students took the ACT in grade 12. The data did not include scores 
on EXPLORE (an ACT test administered in grade 8). 
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slightly higher average test seores in grade 1 than did students in the entire AY2005-gl analysis 
hie. 

The differences noted in the preceding paragraph for the AY2005-gl students also pertain 
to the AY2005-g8 cohort, but to a larger extent. For example, 29% of ah AY2005-g8 students 
were minority, but only 20% of the AY2005-g8 ACT-tested at-grade-level students were 
minority. There were very pronounced differences in the average numbers of infractions, 
punishments, and days absent in grade 8 (0.92, 0.85. and 8.5, respectively, for ah AY2005-g8 
students, versus 0.43, 0.36, and 5.8, respectively, for the AY2005-g8 ACT-tested at-grade-level 
analysis hie). These differences likely rehect the strong relationship of dropping out with 
behavior and attendance (Rumberger (1995), Rumberger & Larson (1998), and Balfanz, Herzog, 
& Mac Iver (2007)). 

Missing Data 

Some of the background variables had missing data. The variables Male, Minority, and 
HomEngl (see Table A-1) had negligible percentages of missing cases (less than one-tenth of one 
percent). The percentage of cases with missing values in the variable FreeMeal differed by year. 
For the AY2005-gl cohort, AY2010 had the largest percentage of missing values (13 percent). 
For the AY2005-g8 cohort, AY2008 had the largest percentage (22 percent). 

Using SAS PROC MI, we created five data sets with missing values imputed from 
nonmissing values. We then estimated models for several outcome variables (InfrTot, PunTot, 
and test scores), and obtained very similar results from ah hve imputation data sets. We 
therefore based ah subsequent analyses on only one of the imputation data sets. 
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General Methodological Features 

We did seven separate analyses in three groups, as defined by the outeome variables 
being predicted; 

Table 1 

Summary of analyses 


Type of outcome 

Analysis file 

Einal 

outcome variable 

Promotion to grade 2 

AY2005-gl cohort analysis 
file 

1. 

Promotion to grade 2 
(InGr2) 

Educational achievement 
in grade 6 

AY2005 -g 1 at-grade-level 
analysis file 

2. 

Grade 6 literacy scale score 
(EtcyScalScr_g6) 



3. 

Attainment of proficiency in 
grade 6 literacy (EtcyPrf_g6) 



4. 

Grade 6 mathematics scale score 
(MathScalScr_g6) 



5. 

Attainment of proficiency in 
grade 6 mathematics (MathPrf g6) 

Educational achievement 
in grades 11/12 

AY2005-g8 ACT-tested 
at-grade-level analysis file 

6. 

ACT Composite score 
(ACT_gll/12) 



7. 

Attainment of all four ACT College 
Readiness Benchmarks (ACT CRB) 


The parenthesized outcome variable names in Table 1 refer to the variables listed in Table A-1 in 
the Appendix. In this section, we describe the common methodological characteristics of the 
predictive modeling in all seven analyses. We describe the particular features and results of each 
analysis in separate sections that follow this one. 
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Predictive models can tell us not only which variables are related to educational 
achievement; they can also tell us how variables from previous years relate to the predictors of 
educational achievement. By assembling relevant predictive models in a network ordered by 
time, we can begin to develop an understanding of how early characteristics of students’ 
educational careers affect later characteristics, including their educational achievement at the 
conclusion of their schooling. With a network of models, we can also estimate how much 
hypothesized improvements that occur early (or consistently) in students’ schooling would 
increase their educational achievement later on. 

In all the analyses, the data are structured hierarchically, with students nested within 
schools.^ The schools in which students were enrolled are likely related to the outcome 
variables, for example, through curriculum (Edvantia, 2005) and other school characteristics 
(Konstantopoulos, 2006). We therefore estimated hierarchical models to predict the outcome 
variables. For interval-scale outcome variables, we estimated linear models with random 
intercepts: 

E{Y,.\X,=x^,...,X^=Xi,) = K+Y,hXik+^j, 

k 

where is the outcome for student i enrolled in school y; Xj,...,x^are values of the 
predictors Xj,...,Xj^ ; are associated fixed effects; and '^i is the value associated with 

school j of a normal random variable with mean 0. We used the MIXED procedure in SAS 
(SAS, 2012) to estimate the hierarchical linear models. 

For dichotomous outcome variables, we estimated hierarchical logistic models: 


ln[ 


Py 

\-p 


] 


bo + YubikX,k + > 


^ Schools are also nested within school districts, but we did not attempt to estimate school district effects in the 
models. 
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where Fl) is the probability that the outcome variable for student i in school j is equal to 1 , and 
the other terms are as defined previously. We used the GLIMMIX procedure in SAS (Bauer & 
Curran, 2006) to estimate the hierarchical logistic models. 

For most of the dependent variables, there are a huge number of potential predictor 
variables, only some of which have relationships strong enough to be detectable with the data. 
Consistent with the exploratory intent of this study, we constructed parsimonious models, using a 
statistical significance cutoff of p<.01, that include only the predictors that we can be reasonably 
confident do not have a zero or trivial relationship with the outcomes. Of course, not including a 
predictor variable in a model does not guarantee that it has no relationship, however small, with 
the outcome variable. Variables not in a model, however, have relationships with the outcome 
variable that we could not reliably distinguish from 0 (in the sense that a 99% confidence interval 
excludes 0). 

Our models include only main effects of the predictor variables; they do not include 
interaction terms. Models with interaction terms would permit more nuanced inferences. With 
interaction terms, for example, we would be able to determine whether behavior is more 
important in predicting the final educational achievement for students with low prior 
achievement than for students with high prior achievement. 

Models with random slopes would also permit more nuanced analyses. For example, we 
might be able to determine whether behavior is more important in predicting achievement at 
some schools than at others. Random effects associated with the behavior variables would also 
adjust for differences among schools in how they collect and code the behavior variables. To 
keep the complexity of these exploratory analyses within reasonable bounds, however, we 
estimated only main-effects random-intercept models. 
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Also missing from our models are teaeher effeets. Nye, Konstantopoulos, and Hedges 
(2004) estimated variances in reading and mathematics achievement in grades 1-3 that could be 
attributed to teachers and schools. For grade 3, they estimated teacher variances of 0.07 and 
0.12, and school variances of 0.02 and 0.05 in the respective two subjects. This result would 
suggest that larger gains in achievement might result from improvements in teachers than from 
improvements in schools. Nevertheless, the teacher variances were small compared to the 
variances jointly associated with background characteristics and prior achievement. The teacher 
variances were also small compared to the student-level residual variances. 

Increasing Promotion to Grade 2 

Children’s experiences in first grade are critical to their later success or failure in school. 
Patterns of behavior and attendance, which we hypothesize influence educational achievement, 
are established then. Furthermore, students’ failure to be promoted to the second grade on time, 
essentially guarantees that they will lag their age cohort throughout their schooling. In our data, 
for example, AY2005-gl students with no disadvantaging background characteristics and with 
average grade 5 literacy and mathematics scores had only a 1% chance of enrolling in grade 6 
during AY2010 if they had not been enrolled in grade 2 on-time in AY2006. 

Modeling Promotion to Grade 2 

From the AY2005-gl cohort analysis file (N=l 8,769), we modeled the probability of 
enrollment in grade 2. The predictor variables were students’ background characteristics, type 
and number of infractions and punishments, number of days absent in grade 1, and first-grade 
literacy and writing scores. 

There were 19 types of infractions (most of which were present in few cases) and 9 types 
of punishments (in addition to a tenth category, “no action taken”). To simplify building the 
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model, we eonsidered only the types of infraetions assoeiated with 1% or more of students: 
insubordination (4%) and disorderly eonduet (11%). Some types of infraetions (e.g., assaults on 
students or staff, or earrying explosives or hand guns) are mueh more serious than 
insubordination and disorderly conduct, and they likely have larger effects on the future 
achievement of the students who commit them. These other types of infractions are rare, 
however, and are unlikely to affect achievement for the broad population of students. 

For the same reason, we included in the model only the types of punishments associated 
with 1% or more of students: in-school suspension (2%), out-of-school suspension (3%), and 
corporal punishment (8%). We used as predictors in the models the frequency of each type of 
infraction or punishment. Note that approximately 11% of the school districts in Arkansas, 
including the largest school district (Little Rock), prohibit corporal punishment in their schools. 
Therefore, the corporal punishment variable in our analyses might be confounded with 
unmeasured district effects. 

The total number of infractions (over all 19 types) and the total number of punishments^ 
are each associated with approximately 10% of first-grade students in AY2005. Therefore, we 
also considered the infraction total frequency and the punishment total frequency as alternative 
predictors. 

Because individual teachers and administrators decide whether to report and punish an 
infraction, the infraction and punishment variables are not standardized measures of 
misbehavior: Misbehavior that one individual reports and punishes might not be reported or 
punished by another individual or at another school. Although we did not do so in this study, 

^ We did not use the frequency of the tenth category “no action taken” in calculating the total number of 


punishments. 
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estimating random effects for the infraction and punishment slopes would address this issue at 
the school level. 

There are also different potential interpretations of the types and frequency of 
punishments. In one interpretation, the imposition of a particular type of punishment is an 
indication of the perceived severity of an infraction. In another interpretation, a punishment is 
simply an intervention which can have an effect, either positive or negative, on days absent and 
test scores. These interpretations are not mutually exclusive; both might be true. 

After standardizing all predictor variables to mean 0 and variance 1, we estimated a 
parsimonious hierarchical logistic model using SAS PROC GLIMMIX and a statistical 
significance level p<.01 (see Table 2). The strongest predictors, as indicated by their 
standardized weights, are the literacy and writing test scores in grade 1. The number of days 
absent is a moderately strong negative predictor, and the number of occurrences of corporal 
punishment is a weak negative predictor. Neither of the infraction counts and none of the 
background variables are in the model. The standard deviation of the random intercept suggests 
moderately strong variation among schools in promotion rate. 

Table 2 

Predictors of Promotion to Grade 2 


Variable 

Coefficient 

Fixed effects (standardized) 

Corporalgl 

-0.08 

DaysAbs_gl 

-0.29 

LtcyNPR gl 

1.55 

WrtgNPR_gl 

0.85 

Random effect SD 

Intercept 

0.78 
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As was noted previously, the absenee of infraetion variables in the model ean be 
interpreted in different ways. One interpretation is that for predicting promotion to grade 2, 
infractions are important only if they are severe enough to result in corporal punishment, but 
their type (insubordination or disorderly conduct) and frequency are statistically irrelevant. 
Another interpretation is that corporal punishment might or might not be related to the perceived 
severity of infractions, but that when it is used, decreases the likelihood of promotion to grade 
two. 

Although the infraction counts and the background variables are not direct predictors of 
the outcome variable, they are indirect predictors, through their associations with the direct 
predictors. Unfortunately, the available data do not indicate the dates of infractions, 
punishments, and absences within a given academic year; therefore, it was not possible to relate 
these variables temporally within year. Instead, we constructed models according to the 
following assumptions: 

• All of the non-test score variables are potential predictors of first-grade test scores. 

• Punishments, infractions, and background variables are potential predictors of days 
absent. 

• Infractions and background variables are potential predictors of punishments. 

• Background variables are potential predictors of infractions. 

The logic behind this structure is that at least one of the punishments (out-of-school suspension) 
obviously increases days absent from school, and punishments result from infractions. The 
background variables are proxies for other, unmeasured, variables that are causes of infractions, 
punishments, and test scores. This chain of relationships is undoubtedly incomplete and 
incorrect in certain respects. For example, corporal punishment might provoke additional 



18 


infractions or aggression (Gershoff, 2002; Durrant & Ensom, 2012). Nevertheless, this simple 
structure provides a way to begin to understand in this data set how students ultimately pass or 
fail first grade. 

Figure 1 on the next page shows the resulting indirect relationships. The solid arrows 
show the direct predictors listed in Table 2. The large dotted arrows show relationships in whieh 
days absent and the discipline variables are predictors. The small dotted arrows show 
relationships in whieh the background variables are predictors. Red arrows denote positive 
relationships, and blue arrows denote negative relationships. To reduee the complexity of the 
diagram, there is no indieation of the strength of individual predictor variables; instead, we 
present model R (or pseudo-R ) for all predietors jointly. 

There are several interesting results: 

• Promotion to grade 2 is predicted moderately well (pseudo-R =.26). Among its direct 
predictors, test scores are predicted weakly (R =.15), and days absent is predicted 
very weakly (R =.05). Frequency of corporal punishment is predicted moderately 
strongly by frequeney of insubordination, frequeney of disorderly conduct, gender, 
and minority status (R =.45). 

• Days absent predicts literacy score, but not writing score. 

• Frequency of corporal punishment predicts the final outcome variable (promotion to 
grade 2), but not days absent or either test score. As one would expect, frequency of 
out-of-school suspension predicts days absent and both test seores. In-sehool 
suspension did not enter into any model. 

• The infraction frequency variables are predieted only very weakly by the background 


variables. 



Background q1 Infractions q1 Punishments q1 Attendance q1 Test scores 
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Figure 1. Statistically significant (p<.01) predictive relatonships related to promotion to grade 2. 
(Blue arrows denote negative relationships; red arrows denote postive relationships.) 
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• The background variables have both direct and indirect relationships with the 
discipline, attendance, and test score variables, but they have no direct relationship 
with promotion to grade 2. In other words, variables such as gender, ethnicity, and 
socio-economic status appear to affect promotion to grade 2 only through their 
relationships with behavior, punishments, attendance, and test scores. 

Simulations 

To estimate the benefits of improved behavior and attendance on promotion to grade 2, 
we did simulations using the prediction models illustrated in Figure 1. The first two scenarios 
involve behavior and attendance: 

1 . No infractions or punishments in grade 1 . 

2. No infractions or punishments, and average or better attendance in grade 1 . 

Note that we are not saying how these improvements would come about; attaining them would 
undoubtedly be difficult. For this reason, the estimated promotion rates associated with the 
scenarios should be thought of as upper bounds to what could plausibly be attained through 
improved behavior and attendance. 

To estimate the effects of improved behavior, for example, we calculated for each student 
the predicted number of days absent, assuming zero infractions and punishments in grade 1 . We 
then used the predicted number of days absent, along with the observed values of the background 
variables and the assumed zero values of infractions and punishments, to calculate predicted test 
scores. We then used the observed values of the background variables, the assumed zero values 
of infractions and punishments, the predicted number of days absent, and the predicted test 
scores, to calculate a probability of promotion to grade 2. We followed a similar strategy 
assuming zero infractions and punishments and average or better attendance. 
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We also estimated benefits for two other seenarios, involving improved prior 
achievement: 

3. Increase grade 1 literacy and writing scale scores by 0.1 SD. 

4. No infractions or punishments, and average or better attendance in grade 1; and 
increase grade 1 literacy and writing scale scores by 0.1 SD. 

In contrast to Scenarios 1 and 2, Scenario 3 is more easily accomplished, at least at small 
scale. Numerous studies show that interventions in early childhood to improve educational 
achievement, behavior, health, and parenting skills result in cognitive skill gains in first grade 
exceeding the hypothesized 0.1 SD (Schweinhart, Barnes, & Weikart (1993); Campbell & 
Ramey (1995); Campbell, Ramey, Pungello, Sparling, & Miller- Johnson, (2002); U.S. 
Department of Health and Human Services (2005)). Interventions that continue into grade 
school and involve significant participation by parents are even more effective (Reynolds, 
Temple, Robertson, & Mann, 2002). Fuller (2007), however, questioned whether the intensive 
interventions behind the benefits of some early childhood programs are practical or affordable at 
large scale, and whether they benefit non-minority and middle-class students. 

Interventions in grade 1 are another way to improve educational achievement. 
Magnuson, Ruhm, and Waldfogel (2007) found that high levels of instruction in the first year of 
school also improve reading skills. 
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Table 3 shows the average estimated probabilities of promotion to grade 2, as ealeulated 
from our models: 

Table 3 

Estimated Rate of Promotion to Grade 2, Given Scenarios of Improved Behavior and Attendance 
in Grade 1 


Scenario in grade 1 


Estimated rate 
of promotion 


(Base rate) 0.95 

1. No infractions or punishments; modeled attendance 0.96 

2. No infractions of punishments and average or better 0.96 

attendance in grade 1 

3. Increase grade 1 literacy and writing scale scores by 0.96 

0.1 SD 

4. No infractions or punishments, average or better 

attendance in grades 1 - 6 , and increase grade 1 0.97 

literacy and writing scale scores by 0.1 SD. 


Improved behavior (Scenario 1), increased behavior and attendance (Scenario 2), and improved 
first-grade achievement (Scenario 3) each increased promotion rate by approximately 0.01. 
Making all three improvements (Scenario 4) increased promotion rate by approximately 0.02. 
Although these increases are small in absolute terms, they are large in relation to the maximum 
possible increase (0.05). 

Improving Educational Achievement in Grade 6 

Students in the AY2005-gl cohort who were promoted on schedule were in grade 6 
during AY2010. Arkansas administers to its sixth-grade students two criterion-referenced 
achievement tests, in literacy and mathematics, and reports both scale scores and proficiency 



23 


levels, based on state standards. We developed models to predict the scale scores and attainment 
of proficiency. 

Models for Grades 1-6 

From the AY2005-gl at-grade-level analysis file (N=14,420), we modeled behavior, 
attendance, and educational achievement at each grade level (1 - 6). To simplify the modeling, 
we used the total number of infractions (summed over all types of infractions) and the total 
number of punishments (summed over all types of punishments) as predictors. We structured the 
chain of predictive models according to the scheme in Table 4 (see next page): In a given 
academic year, test scores are preceded by days absent, which are preceded by punishments, 
which are preceded by infractions, which are preceded by background variables and by all types 
of variables from prior academic years. We transformed all predictor variables to have mean 0 
and variance 1 . 
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Table 4 

Model Structure for Chained Predictions in Grades 1-6 


Predictor 

variables 



Dependent variables 




Grade 1 



Grades 2-6 


Total 

pun. 

Days abs. 

Test 

scores 

Total 

infr. 

Total 

pun. 

Days abs. 

Test 

scores 

Background 








variables 

X 

X 

X 

X 

X 

X 

X 

Current vear nredictor variables 






Total in fir. 

X 

X 

X 

. . . 

X 

X 

X 

Total pun. 

. , . 

X 

X 

. . , 

. . , 

X 

X 

Days abs. 

. . , 

. . . 

X 

. , . 

. , . 

. . . 

X 

Prior vears’ nredictor variables 






Total infr. 

. . , 

. . . 

. . . 

X 

X 

X 

X 

Total pun. 

, , , 

. . . 

. . , 

X 

X 

X 

X 

Days abs. 

. . . 

. . . 

. , . 

X 

X 

X 

X 

Test scores 

. . , 

. . , 

, . . 

X 

X 

X 

X 


Note: An X indicates that a predictor variable (row) is eligible to appear in the model for the 
indicated dependent variable (column). 


Table A-3 in the appendix summarizes the resulting models. 

The models for infraction frequencies have moderate strength (R = .14 - .25). In grades 
2-3, the principal predictors are infraction frequencies from the preceding year. In grades 4-6, 
the principal predictors are prior year punishment frequencies; prior year literacy scores are weak 
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negative predictors. There is moderate variation in the intercept among schools (SD=0.23 - 
0.35). This variation might reflect actual differences in misbehavior, or it might reflect 
differences in reporting. One way to investigate causes of this variation would be to interview 
officials at schools with large or small estimated residuals. 

The models for punishment frequencies are very strong (R =.90 - .94), reflecting the 
obvious dependence of punishment frequencies on current year infraction frequencies. Prior 
year punishment frequencies are also in the models, but are much less important. Prior year 
Writing or Literacy scores are weak negative predictors. Males and minority students have 
slightly larger punishment frequencies, given the other variables in the models, than female or 
non-minority students. This result is consistent in direction with that reported by Pfleger and 
Wiley (2012).^ Variation among schools in the intercept is small (SD=0.08 - 0.12). 

The models for days absent in grades 2-6 are moderately strong (R =.34 - .39); the 
principal predictor is days absent in the preceding year. Current year punishment frequency is a 
weak predictor. Current year infraction frequency is not in the model, given its strong 
correlation with punishment frequency. The intercept standard deviations are moderately large 
(0.15 - 0.30). The grade 1 model for days absent is weak (R =.04), given that it has no variable 
measuring days absent in the preceding year. 

The models for educational achievement test scale scores in grades 2-6 are strong 
(R =.53 - .71). Not surprisingly, the principal predictors are prior year test scores. Prior year 
punishment frequencies and days absent are weak, but consistent, predictors at all grades. The 
coefficients for days absent (-0.01 to -0.06) are slightly smaller in magnitude than those reported 


* Because of limitations in their data, Pfleger and Wiley were not able to control for infraction frequency in 
predicting punishment frequency. 
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by Gottfried (2010, 2011). The intereept standard deviations are small to moderate (0.14 - 0.25), 
suggesting unmeasured school effects. 

In the models for educational achievement in grades 2 and 3, the intercept standard 
deviations range from 0.15 to 0.25, which correspond to intercept variances of approximately 
0.02 to 0.06. Nye, Konstantopoulos, and Hedges (2004) estimated three-level models (students 
within classrooms within schools) of reading and mathematics test score data from grades 2 and 
3 in Tennessee. Like our models, their models included background characteristics and prior 
achievement as student-level predictors. They reported between-school intercept variances of 
approximately 0.02 to 0.05. Thus, both studies show that schools are associated with 
considerably less than 10% of the total variance in test scores in grades 2 and 3. 

We also predicted attainment of state achievement standards at the proficient or higher 
level in grade 6 literacy and mathematics, with hierarchical logistic regression models.^ As in 
the linear models for grade 6 scale scores, the principal predictors in the proficiency models are 
grade 5 scale scores. Grade 5 punishment frequencies and days absent are also in the proficiency 
models. 

Summary. For predicting educational achievement in grades 2-6, prior achievement is 
by far more important (as measured by its standardized regression weight) than any other class of 
variables considered in this study. This result holds at all grade levels. Nevertheless, behavior 
and attendance consistently contribute in small amounts to educational achievement; 


^ We used the same subset of data to estimate the proficiency models that we used to estimate the scale score 
models. Alternatively, one could include in the proficiency models data for students who were retained in grades 
1-5, assuming that they would not have met the mathematics proficiency standards for grade 6. Doing so would 
have likely improved the fit of the models. To avoid making additional assumptions and to simplify interpreting the 
results, however, we used the same subset of data to estimate both types of models. 
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furthermore, educational achievement contributes in small amounts to predicting future behavior 
and attendance. 

Simulations 

A principal goal of this study is to estimate the potential benefits of improved behavior, 
attendance, and prior achievement on achievement in grade 6. To do this, we did simulations 
using the prediction models in Table A-3. There were four scenarios: 

1 . No infractions or punishments in grades 1-6. 

2. No infractions or punishments and average or better attendance in grades 1-6. 

3. Increase grade 1 literacy and writing scale scores by 0.1 SD. 

4. No infractions or punishments and average or better attendance in grades 1 - 6 ; and 
increase grade 1 literacy and writing scale scores by 0.1 SD. 

The first two scenarios involve improvements in behavior and attendance only. Scenarios 3 and 
4, involving increased test scores in grade 1 , are intended to provide a comparison to the first two 
scenarios. Scenarios 3 and 4 could represent the effects of interventions in grade 1 alone, the 
effects of interventions before grade 1 , or the effects of interventions before and during grade 1 . 
By structuring the simulations around the chained predictive models summarized in Table 4, one 
can estimate how a change in a particular kind of variable (e.g., infractions) at one time affects 
all other variables (infractions, punishments, attendance, test scores) at later times. 

Attaining zero infractions and punishments for every student throughout grades 1 - 6 is a 
laudable, but unrealistic goal. Reducing absence to the current average would also likely be very 
difficult to attain. Therefore, the estimated benefits associated with Scenarios 1 and 2 should be 
thought of as upper bounds to what can feasibly be attained. Lassen, Steele, and Sailor (2006), 
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Skaggs and Bodenhom (2006), and What Works Clearinghouse (2012) deseribe programs that 
improve behavior and attendance, though not to this ideal level. 

In Scenario 1, we calculated for each student predicted days absent and test scores, 
assuming zero infractions and punishments in each grade and observed values of the background 
variables. We also used the predicted days absent and test scores in a given grade to predict days 
absent and test scores in higher grades. 

We followed a similar strategy in Scenario 2, assuming zero infractions and punishments 
and average or better attendance. The number of days absent for a particular student at a given 
grade level was taken to be the minimum of the student’s predicted value and the average 
observed number of days absent over all students. 

We next repeated Scenarios 1 and 2, assuming different starting grades (2 - 6) for the 
improvements. For example, we calculated the benefit of zero infractions and punishments, and 
average or better attendance, beginning in grade 2, but assuming observed infraction and 
punishment frequencies and days absent in grade 1. These simulations provide information on 
how soon improvements need to begin to realize benefits. 

In Scenario 3, we assumed that each student’s literacy and writing scores in grade 1 were 
0.1 SD higher than the observed scores. As we have already noted, this improvement is more 
feasibly attained than the improvements hypothesized in Scenarios 1 and 2. Scenario 4 involves 
all the improvements: behavior, attendance, and prior achievement. 

Although school ID was available in this study, we did not estimate benefits associated 
with scenarios involving improvements in schools. Some rough idea of potential benefits, 
however, can be had by examining the variances associated with student-level characteristics and 
intercepts. In grade 6, the variances associated with student-level characteristics are 0.71 and 
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0.69 for literacy and mathematics, respectively; the residual student-level variances are 0.27 for 
both areas, and the intercept variances are 0.02 and 0.04, respectively. This would suggest that 
larger improvements in test scores would result from interventions on characteristics other than 
schools. Of course, school characteristics are also related to achievement in grades 1 - 5; it is 
plausible that consistent improvement in schools throughout these earlier grades would result in 
greater improvement in grade 6 achievement than would be suggested by the intercept variance 
in the model for grade 6 achievement. 

Figures 2 and 3, on pages 30-31, show the increase in average predicted standardized 
grade 6 scores in literacy and mathematics, respectively, by scenario. Figures 4 and 5, on pages 
34-35, show comparable results for the estimated increase in proficiency rates. In each graph, 
the increase in grade 6 achievement (vertical axis) is plotted against the starting grade of the 
hypothesized prior improvement in behavior or attendance (horizontal axis). Each graph also has 
an indication of the increase in grade 6 achievement associated with improved test scores in 
grade 1 (Scenarios 3 and 4). 

The benefit of all types of improvement is modest. In Figure 2, for example, improving 
behavior starting in grade 1 (Scenario 1) increases grade 6 literacy scores by 0.03 SD. 
Improving both behavior and attendance starting in grade 1 (Scenario 2) results in an increase of 
0.05 SD. Starting improved behavior and attendance as late as grade 3 would preserve about 
three-quarters of the benefits associated with starting in grade 1 . Starting improved behavior and 
attendance in grade 4 or grade 5 would preserve about half of the benefits. 

Increasing grade 1 literacy and writing scores by 0.1 SD (Scenario 3) increases average 
predicted grade 6 literacy score by somewhat less than 0.04 SD; this result indicates a decay over 
time in the effects of early interventions on educational achievement. Dougherty (2010) reported 
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similar results when predicting test scores of middle school students from the school-level 
residuals associated with the elementary schools that they had previously attended. 

Making all improvements (Scenario 4), however, increases average predicted grade 6 
literacy score by approximately 0.09 SD, a level nearly comparable to the hypothesized initial 
improvement (0.1 SD) in educational achievement. This result suggests that to sustain the initial 
benefits of early childhood programs, we must follow them with continued interventions in grade 
school. In this study, the hypothesized continued interventions pertained only to behavior and 
attendance; continued improvement in educational achievement directly would, of course, yield 
greater gains by grade 6. 

Similar patterns pertain to improvements in grade 6 mathematics score (Figure 3), 
although the amount of improvement is smaller than for literacy. Starting improved behavior 
and attendance (Scenario 2) as late as grade 3 preserves nearly all of the benefits for mathematics 
achievement that are associated with starting in grade 1 . Increasing grade I literacy and writing 
scores by 0.1 SD (Scenario 3) increases average predicted grade 6 mathematics score by less 
than 0.04 SD, indicating a decay over time in the effects of early interventions. Making all 
improvements (Scenario 4), however, increases average predicted grade 6 mathematics score by 
approximately 0.08 SD, a level more nearly comparable to the hypothesized initial improvement 
(0.1 SD) in educational achievement. 



Prior ach., behavior, & attendance 
(Seen. 4) 
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Figure 3. Average standard deviation increase in grade 6 mathematics score, by type and starting grade of improvement. 
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Figures 4 and 5 show estimated inereases in proficiency rates for literacy and 
mathematics, respectively. Improving behavior and attendance beginning in grade 1 and 
sustaining it thereafter (Scenario 2) increases the literacy proficiency rate by 0.03 beyond the 
base rate of 0.73. The corresponding improvement in mathematics proficiency rate is 0.04 
beyond its base rate of 0.77. By way of comparison, increasing first-grade test scores by 0.1 SD 
(Scenario 3) improves proficiency rates by about only 0.02 in either subject. Making a one-time 
improvement in first-grade achievement and sustained improvements in behavior and attendance 
(Scenario 4) improves proficiency rates by approximately 0.05. 
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Figure 4. Average increase in grade 6 literacy proficiency rate, by type and starting grade of improvement. 


Prior ach., behavior, & attendance 
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Improving College Readiness 

Our analyses of the AY2005-gl eohort focused on students’ literacy and mathematics 
skills in grade 6 and on their proficiency relative to state standards. Our analyses of the 
AY2005-g8 cohort, in contrast, focused on students’ readiness to take college-level courses when 
they graduate from high school. The outcome variables for these analyses were ACT_gll/12, 
the rounded average of the four ACT scores in English, mathematics, reading, and science (ACT, 
2007), and ACT_CRB (a variable indicating whether a student’s four ACT scores met all of the 
associated ACT College-Readiness Benchmarks). We used the most recent scores of students 
who took the ACT more than once. 

The ACT College Readiness Benchmark in a subject area is the score at which a student 
enrolled in a typical postsecondary institution has approximately a 50% chance of earning a B or 
higher grade in a related first-year college course (Allen & Sconing, 2005; ACT, 2010). The 
ACT subject areas, their associate Benchmark scores and related college courses are: English-18 
(English Composition), Mathematics-22 (College Algebra), Reading-21 (social sciences 
courses), and Science-24 (Biology). In the 2009 high school graduating class (which 
corresponds to the last year of the AY2005-g8 cohort data), approximately 23% of ACT-tested 
students nationally and 18% of ACT-tested students in Arkansas attained all four Benchmarks. 

Arkansas’ state-administered tests for middle-school and high-school students are more 
varied than those for students in grades 1-6. Eighth-grade students take criterion-referenced 
tests in literacy and mathematics, analogous to those geared to grades 3-7. Ninth-grade students 
take norm-referenced tests in literacy, mathematics, and writing. Students in grade 1 1 take an 
end-of-course test in English/Eanguage Arts (denoted here as EEA gl 1). Students in grades 9 - 
12 can also take end-of-course tests in other subjects, such as algebra, geometry, and biology. 
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Where possible, we aligned the measures of educational achievement in grades 8-11 
with the principal outcome variables, ACT_gll/12 and ACT CRB; For grade 8, we used 
LitMath_g8, an equal SD-weighted average score of the two criterion-referenced test scores. For 
grade 9, we used LitMathWrtg_g9, an equal SD-weighted average of the three norm-referenced 
test scores. For grade 1 1, we used the score on ELA gl 1, the only state-sponsored test taken by 
most students at that grade level. 

No individual state-sponsored tests are taken by most students in grade 10. To measure 
educational achievement in grade 10, we used the PLAN Composite score (denoted here as 
PLAN glO), the rounded average of the four PLAN scores in English, mathematics, reading, and 
science. The content of PLAN (ACT, 2011b) is aligned with that of the ACT, and most 
students in Arkansas who take the ACT also take PLAN. 

Models for Grades 8-12 

We developed models for behavior, attendance, and educational achievement for the 
2005-g8 ACT-tested at-grade-level analysis file (N=10,196). The chaining structure of the 
models is analogous to that for grades 1-6 (see Table 4)**^. Table A-4 in the appendix 
summarizes the resulting models. 

The models for infraction frequencies are moderately strong (R = .18 - .32). The 
principal predictors vary by grade. In grades 9 and 11, the principal predictors are infraction 
frequencies from the preceding year; in grades 10 and 12, the principal predictors are punishment 
frequencies from the preceding year. There is moderate variation in the intercept among schools 
(SD=0.16 - 0.30). As with grades 1 - 6, we do not know whether this variation reflects actual 
differences in misbehavior, or whether it reflects differences in reporting. 

For students who took the ACT in grade 11, the grade 12 infraction frequencies, punishment frequencies, and 
number of days absent are actually “postdictors” of the ACT outcome variables. 
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2 

The models for punishment frequeneies are very strong (R =.87 - .96), refleeting the 
obvious dependenee of punishment frequeneies on infraction frequencies in the current year. 
Punishment frequencies in the prior year are also in the models, but are much less important. 
Variation among schools in the intercept is small (SD=0.06 - 0.1 1). 

The models for days absent in grades 9-12 are moderately strong (R =.32 - .38); the 
principal predictors are days absent in preceding years. The intercept standard deviations are 
fairly large (0.30 - 0.45). The grade 8 model is weak (R =.04), because it has no variable 
measuring days absent in the preceding year. 

The models for educational achievement scale scores in grades 9 - 1 1 are strong (R =.60 - 
.72). The model for grade 8 test score is weaker (R =.14), because it does not include test scores 
from the preceding year. The principal predictors are test scores from previous years (weights 
0.69 - 0.84). Current year days absent is a weak predictor at grades 8, 9, and 11. 

The model for predicting ACT_gll/12 is also strong (R =.79). Not surprisingly, the 
principal predictors of ACT_gll/12 measure prior achievement: PLAN glO (weight 0.60) and 
ELA gl 1 score (weight 0.31). PLAN glO is a stronger predictor than ELA gl 1 because of the 
similarity of the PLAN and ACT Composite scores in their subject area coverage (English, 
Mathematics, Reading, and Science) and their targeted content standards. 

The regression weights associated with days absent in the ACT_gll/12 model are -0.03 
(for grade 12) and -0.02 (for grade 11); when combined, they are in line with the 0.05 result 
reported by Gottfried (2010, 2011), Our ACT_gll/12 model does not include infraction 
frequency as a predictor, but the regression weight for punishment frequency is -0.02. This 
result is considerably smaller than the magnitude 0.11 weight reported by Lassen, Steele, and 
Sailor (2006), and might be due to the latter study not including prior achievement in the model. 
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The ACT_gll/12 model has an intereept standard deviation of 0.09, whieh eorresponds 
to an intereept variance of approximately 0.01. Sawyer (2008) estimated two-level models 
(students within schools) for the ACT English, Mathematics, Reading, and Science scores, using 
background characteristics, EXPLORE (grade 8) scores, and high school course work and grades 
as predictors. He obtained intercept variances, which when standardized, range from 0.01 to 
0.03. The slightly larger variances in the Sawyer (2008) study might be due in part to their being 
based on a broader sample (more than one state) and on different covariates. In both studies, 
however, the variance associated with high schools is considerably smaller than 0.05. 

We also predicted attainment of the ACT College Readiness Benchmarks, using a 
hierarchical logistic model. As in the ACT_gll/12 model, the principal predictors are 
PLAN glO and ELA_1 1 scores, but days absent in grades 10 and 1 1 are also in the model. 

Summary. The results for predicting educational achievement in grades 8 and higher are 
similar to those for predicting achievement in grades 2-6: Prior achievement is the most 
important variable, but behavior and attendance consistently contribute in small amounts to 
educational achievement. Moreover, educational achievement contributes in small amounts to 
predicting future behavior and attendance. 
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Simulations 

In a manner similar to what we did for grades 1 - 6, we estimated the potential benefits of 
various seenarios of improved behavior, attendanee, and prior aehievement on students’ 
readiness for eollege as measured by the variables ACT_gll/12 and ACT CRB. There were 
four scenarios: 

1. No infractions or punishments in grades 8 - 12. 

2. No infractions or punishments, and average or better attendance in grades 8 - 12. 

3. Increase grade 8 literacy/writing average score by 0.1 SD. 

4. No infractions or punishments and average or better attendance in grades 8 - 12; and 
increase grade 8 literacy/writing average score by 0.1 SD. 

As we noted about the simulations for grades 1-6, Scenarios 1 and 2 are idealistic; they provide 

upper bounds to the benefits that could be obtained from improving behavior and attendance. 

Given the results from grades 1-6, however. Scenario 3 is more easily attainable. In estimating 

the benefits for these scenarios, we followed the same chaining structure described in Table 4. 

It is unlikely that waiting until grade 8 to begin interventions in behavior, attendance, and 

prior achievement would achieve the improvements hypothesized in the scenarios; interventions 

would need to begin well before grade 8. Moreover, assessment and interventions throughout 

middle school and high school, such as those enabled by ENGAGE (ACT, 2012), EXPEORE, 

and PLAN, would likely be needed to sustain the improvements. 

Figure 6 on page 42 shows the simulation results for the average ACT Composite score. 

Figure 7 on page 43 shows comparable results for the estimated proportion of students attaining 

all four ACT College Readiness Benchmarks. Each graph shows the increase, beyond the 

*’ ENGAGE Grades 6-9 measures psychosocial characteristics that predict academic performance and persistence in 
high school. It has ten scales organized in the domains of motivation, social engagement, and self-regulation. 
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observed average of students in AY2009, plotted against the starting grade of the improvement. 
On each graph there is also an indication of the increased achievement associated with Scenarios 
3 and 4. 

The benefit of all types of improvement is modest. In Figure 6, for example, improving 
behavior in grades 8-12 increases ACT Composite scores by 0.03 SD. Improving both behavior 
and attendance in grades 8-12 results in an increase of slightly less than 0.04 SD. By way of 
comparison, increasing grade 8 literacy/mathematics average score by 0.1 SD (Scenario 3) 
increases average ACT Composite score by 0.05 SD. Making all improvements (Scenario 4) 
increases average predicted ACT Composite score by slightly less than 0.09 SD. 

Figure 7 shows the estimated increase in the proportion of students attaining all four ACT 
College Readiness Benchmarks. Improving behavior and attendance beginning in grade 8 and 
sustaining the improvement thereafter (Scenario 2), increases the attainment rate by somewhat 
less than 0.02 beyond the base rate of 0.19. A one-time improvement in grade 8 achievement 
(Scenario 3) results in an increase of 0.03. Improving behavior, attendance, and prior 
achievement (Scenario 4) results in an increase of 0.04. 

As in the grade 6 analyses, we did not estimate benefits associated with scenarios 
involving improved schools. Sawyer (2008) found in a simulation study that improving below- 
average high schools (as measured by their Empirical Bayes estimates) to the average would 
improve ACT College Readiness Benchmark attainment rates by 0.02 or less, depending on 
subject area. 



Prior ach., behavior, & attendance 
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Figure 6. Average standard deviation increase in ACT Composite score, by type and starting grade of improvement. 
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Figure 7. Average increase in ACT College Readiness Benchmark attainment rate, by type and starting grade of improvement. 
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Discussion 

There are many variables that plausibly eontribute to edueational aehievement, but it is 
not feasible to eolleet data on all of them, espeeially over an extended period of time. The 
predietor variables available in this study involve student baekground characteristies, sehool ID, 
behavior, attendanee, and prior aehievement. Among the potentially important predietor 
variables not ineluded are student psychosoeial eharaeteristics, parental involvement, variables 
related to teaeher effectiveness, school characteristics (other than school ID), and neighborhood 
characteristics. 

We studied three groups of students at schools that reported data on infractions, 
punishments, and attendance: 

• all first-grade students (to learn characteristics that predict promotion to grade 2), 

• all sixth-grade students who were at expected grade level (to learn characteristics that 
predict achievement on state proficiency tests), and 

• all twelfth-grade students who were at expected grade level (to learn characteristics 
that predict academic readiness for college, as measured by the ACT Composite score 
and the ACT College Readiness Benchmarks). 

In the last two analyses, we did not attempt to make inferences about the educational 
achievement of students who had been held back a grade, moved out of the Arkansas public 
school system, or dropped out of high school entirely. These students would be an important 
focus of future research. 

We learned that educational achievement in grade 1 (as measured by scores on state tests) 
is strongly and positively related to promotion. Corporal punishment and absenteeism in grade 1 
are negatively related to promotion. Infraction frequencies and background characteristics 
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predict promotion to grade 2 only indirectly, through their effects on corporal punishment, 
absenteeism, and test scores. 

We also learned that prior achievement dominates background characteristics, behavior, 
and attendance in predicting long-term achievement. Moreover, the benefit of plausible 
improvements in prior achievement decays over time: Continued interventions (in this study, 
pertaining to behavior and attendance) are needed to sustain the initial improvement. Given the 
research results of others, improved curriculum, teaching, and parental support are also 
beneficial (and might be required) for sustained gains in achievement. Interventions to improve 
psychosocial characteristics, such as those measured by ENGAGE, could also provide important 
additional benefits. No single intervention or reform, however, seems likely to achieve sustained 
and large-scale improvement. Such an outcome will require persistent efforts on multiple fronts, 
each of which will contribute incrementally to the goal. 
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Table A-1 

Variables in the Study 


Variable name 

Deseription 

Backsround characteristics 

Male 

Gender (Female=0, Male=I) 

Minority 

Underserved minority (Caueasian Am., Asian Am. = 0; all others =1) 

HomEngl 

Primary language spoken at home (English =1) 

FreeMeal 

Eree / redueed luneh (Eligible=l) 

Title 1 

Title I (Eligible=l) 

EeonDis 

State-designated as eeonomieally disadvantaged (1) 

Enrollment variables 

SehoolID 

State ID of sehool in whieh student was enrolled 

GrdLvl 

Grade level in whieh student was enrolled (1-6,8-12) 

Discipline variables 

DisordCondgl 

Number of instanees of disorderly eonduet in grade 1 

Insub gl 

Number of instanees of insubordination in grade 1 

Corporalgl 

Number of instanees of eorporal punishment in grade 1 

OutSehlSuspgl 

Number of instanees of out-of-sehool suspension in grade 1 

InfrTot gl 

Total number of infraetions reported in grade 1 

InfrTot g2 

Total number of infraetions reported in grade 2 

InfrTot g3 

Total number of infraetions reported in grade 3 

InfrTot g4 

Total number of infraetions reported in grade 4 

InfrTot g5 

Total number of infraetions reported in grade 5 

InfrTot g6 

Total number of infraetions reported in grade 6 

InfrTot g8 

Total number of infraetions reported in grade 8 

InfrTot g9 

Total number of infraetions reported in grade 9 

InfrTot glO 

Total number of infraetions reported in grade 10 

InfrTot gll 

Total number of infraetions reported in grade 1 1 

InfrTot gI2 

Total number of infraetions reported in grade 12 

PunTotgl 

Total number of punishments reported in grade 1 

PunTot_g2 

Total number of punishments reported in grade 2 

PunTotgS 

Total number of punishments reported in grade 3 

PunTot_g4 

Total number of punishments reported in grade 4 

PunTot_g5 

Total number of punishments reported in grade 5 

PunTotgb 

Total number of punishments reported in grade 6 

PunTotgS 

Total number of punishments reported in grade 8 

PunTot_g9 

Total number of punishments reported in grade 9 

PunTotglO 

Total number of punishments reported in grade 10 

PunTotgll 

Total number of punishments reported in grade 1 1 

PunTot_gI2 

Total number of punishments reported in grade 12 


(continued on next page) 
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Table A-1 
(continued) 


Variable name 

Description 

Attendance 

DaysAbs_gl 

Days absent in grade 1 

DaysAbs_g2 

Days absent in grade 2 

DaysAbs_g3 

Days absent in grade 3 

DaysAbs g4 

Days absent in grade 4 

DaysAbs_g5 

Days absent in grade 5 

DaysAbs_g6 

Days absent in grade 6 

DaysAbs_g8 

Days absent in grade 8 

DaysAbs_g9 

Days absent in grade 9 

DaysAbsglO 

Days absent in grade 10 

DaysAbs gll 

Days absent in grade 1 1 

DaysAbs gl2 

Days absent in grade 12 

Scores on state-sponsored tests 

Scale for NRT scores 

in gl - g2: - 1 - 99 

Scale for CRT scores 

ing3-g8: -20-990 

Scale for NRT average score in g9: -1-99 

Scale for ELA gll score: 30-315 

LtcyNPR gl 

Literacy NRT national percentile rank in grade 1 

WrtgNPR_gl 

Writing NRT national percentile rank in grade 1 

LtcyNPR g2 

Literacy NRT national percentile rank in grade 2 

WrtgNPR_g2 

Writing NRT national percentile rank in grade 2 

LtcyScalScr _g3 

Literacy CRT score in grade 3 

MathScalScr g3 

Mathematics CRT score in grade 3 

LtcyScalScr _g4 

Literacy CRT score in grade 4 

MathScalScr g4 

Mathematics CRT score in grade 4 

LtcyScalScr _g5 

Literacy CRT score in grade 5 

MathScalScr g5 

Mathematics CRT score in grade 5 

LtcyScalScrgb 

Literacy CRT score in grade 6 

MathScalScr g6 

Mathematics CRT score in grade 6 

LitMath g8 

SD-weighted average Literacy and Mathematics CRT score in grade 8 

LitMathWrtg g9 

SD-weighted average Literacy, Mathematics, and Writing NRT score in 

ELAgl 1 

grade 9 

English/Language Arts end-of-course test score in grade 1 1 


(continued on next page) 




57 


Table A-1 
(continued) 


Variable name 

Deseription 

Scores on ACT-sponsored tests 

PLAN glO 

PLAN Composite seore in grade 10 (seale: 1 - 30) 

ACT_gll/12 

ACT Composite seore in grades 1 1/12 (seale: 1 - 36) 

Success indicators 


InGr2 

Enrolled in grade 2 in AY2006 (Yes=l) 

LteyPrf g6 

At or above profieient level on grade 6 Literaey test (Yes=l) 

MathPrf g6 

At or above profieient level on grade 6 Mathematies test (Yes=l) 

ACT CRB 

Meets all four ACT College Readiness Benehmarks (Yes=l) 




Table A-2 

Summary of Variables, by Analysis File 
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Table A-3 

Prediction of Infractions, Punishments, Attendance, and Achievement in gl-g6 
(Parsimonious models; p<.01) 


Outcome variable 

Intercept 

SD 

Level- 1 

Level- 1 

predictor variable 

Fixed effect 

Infractions 





InfrTot g2 

.29 

.20 

Male 

.07 




Minority 

.05 




HomEngl 

.05 




FreeMeal 

.02 




InfrTot gl 

.31 




PunTotgl 

.08 

InfrTot g3 

.30 

.17 

Male 

.07 




Minority 

.05 




HomEngl 

.04 




FreeMeal 

.04 




InfrTot g2 

.27 




PunTot_g2 

.11 

InfrTot g4 

.35 

.14 

Male 

.07 




Minority 

.03 




HomEngl 

.05 




FreeMeal 

.03 




PunTot_g3 

.32 




EtcyScalScr_g3 

-.05 

InfrTot g5 

.33 

.17 

Male 

.07 




Minority 

.06 




HomEngl 

.04 




FreeMeal 

.04 




PunTot_g4 

.34 




EtcyScalScr_g4 

-.04 

InfrTot g6 

.23 

.25 

Male 

.08 




Minority 

.06 




HomEngl 

.05 




FreeMeal 

.03 




InfrTot g5 

.13 




PunTot_g5 

.29 




EtcyScalScr_g5 

-.06 


(continued on next page) 
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Table A-3 
(continued) 


Outcome variable 

Intercept 

SD 

Level- 1 

Level- 1 

predictor variable 

Fixed effect 

Punishments 





PunTotgl 

.08 

.93 

Male 

.02 




Minority 

.01 




HomEngl 

.01 




FreeMeal 

.01 




InfrTot gl 

.96 

PunTot_g2 

.09 

.94 

Male 

.01 




FreeMeal 

.01 




PunTot gl 

.03 




WrtgNPRgl 

-.01 




InfrTot g2 

.9S 

PunTotgS 

.08 

.94 

Male 

.02 




FreeMeal 

.01 




PunTot g2 

.03 




WrtgNPR_g2 

-.01 




InfrTot g3 

.96 

PunTot_g4 

.08 

.94 

Male 

.01 




Minority 

.01 




PunTot_g3 

.04 




FtcyScalScr_g3 

-.01 




InfrTot g4 

.96 

PunTotgS 

.12 

.90 

Male 

.02 




Minority 

.01 




HomFngl 

.01 




FreeMeal 

.01 




PunTot_g4 

.04 




FtcyScalScr_g4 

-.02 




InfrTot g5 

.93 

PunTot_g6 

.12 

.90 

Male 

.02 




PunTotgS 

.OS 




DaysAbsgS 

.01 




FtcyScalScrgS 

-.02 




InfrTot g6 

.93 


(continued on next page) 
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Table A-3 
(continued) 


Outcome variable 

Intercept 

SD 

Level- 1 

Level- 1 

predictor variable 

Fixed effect 

Attendance 





DaysAbsgl 

21 

.04 

Minority 

-.14 




HomEngl 

.03 




FreeMeal 

.19 

DaysAbs_g2 

.15 

.39 

Male 

-.03 




Minority 

-.05 




DaysAbsgl 

.61 




LtcyNPR gl 

-.04 




PunTot_g2 

.03 

DaysAbsgS 

.18 

.36 

Minority 

-.08 




HomEngl 

.02 




FreeMeal 

.06 




DaysAbs g2 

.58 




WrtgNPR_g2 

-.03 




PunTot_g3 

.04 

DaysAbs g4 

.20 

.39 

Minority 

-.08 




FreeMeal 

.06 




PunTot_g3 

-.02 




DaysAbs_g3 

.58 




FtcyScalScr_g3 

-.02 




PunTot_g4 

.06 

DaysAbsgS 

.24 

.34 

Minority 

-.09 




HomEngl 

.02 




FreeMeal 

.08 




DaysAbs g4 

.54 




FtcyScalScr_g4 

-.04 




PunTotgS 

.06 

DaysAbs_g6 

.30 

.37 

Minority 

-.08 




FreeMeal 

.07 




PunTotgS 

-.02 




DaysAbsgS 

.55 




MathScalScrgS 

-.04 




PunTot_g6 

.12 


(continued on next page) 
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Table A-3 
(continued) 


Outcome variable 

Intercept 

SD 

Level- 1 
R^ 

Level- 1 

predictor variable 

Fixed effect 

Educational achievement 





LtcyNPR gl 

.29 

.14 

Male 

-.09 




Minority 

-.17 




HomEngl 

.06 




FreeMeal 

-.16 




PunTotgl 

-.06 




DaysAbsgl 

-.07 

WrtgNPRgl 

.31 

.15 

Male 

-.14 




Minority 

-.17 




HomEngl 

.03 




FreeMeal 

-.16 




PunTotgl 

-.07 




DaysAbsgl 

-.08 

LtcyNPR g2 

.15 

.65 

Minority 

-.07 




HomEngl 

.03 




FreeMeal 

-.05 




EtcyNPR gl 

.59 




WrtgNPRgl 

.20 

WrtgNPR_g2 

.21 

.58 

Male 

-.06 




Minority 

-.07 




HomEngl 

-.02 




FreeMeal 

-.05 




EtcyNPR gl 

.34 




WrtgNPRgl 

.39 




PunTotgl 

-.02 




DaysAbsgl 

-.04 

LtcyScalScrgS 

.21 

.60 

Male 

-.09 




Minority 

.02 




HomEngl 

-.03 




FreeMeal 

-.04 




PunTot_g2 

-.02 




EtcyNPR g2 

.50 




WrtgNPR_g2 

.30 




PunTot_g3 

-.04 




DaysAbs_g3 

-.05 


(continued on next page) 
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Table A-3 
(continued) 


Intercept 

Outcome variable SD 

Level- 1 

Level- 1 

predictor variable 

Fixed effect 

Educational achievement (continued) 




MathScalScrgS .25 

.53 

Male 

.04 



Minority 

-.07 



HomEngl 

-.05 



FreeMeal 

-.02 



PunTot_g2 

-.02 



LtcyNPR g2 

.37 



WrtgNPR_g2 

.37 



PunTot_g3 

-.03 



DaysAbs_g3 

-.06 

LtcyScalScr_g4 .17 

.68 

Male 

-.08 



Minority 

-.03 



FreeMeal 

-.03 



FtcyScalScr_g3 

.55 



MathScalScr_g3 

.29 



DaysAbs g4 

-.01 

MathScalScr g4 .19 

.66 

Male 

.03 



Minority 

-.06 



HomFngl 

-.02 



FreeMeal 

-.03 



FtcyScalScr_g3 

.23 



MathScalScr g3 

.61 



DaysAbs g4 

-.02 

LtcyScalScrgS .15 

.71 

Male 

-.03 



Minority 

-.03 



FreeMeal 

-.04 



FtcyScalScr_g4 

.61 



MathScalScr g4 

.23 



PunTot_g5 

-.03 



DaysAbs_g5 

-.02 

MathScalScr_g5 .23 

.67 

Male 

.03 



Minority 

-.04 



HomFngl 

-.03 



FreeMeal 

-.02 



FtcyScalScr_g4 

.27 



MathScalScr_g5 

.56 



PunTot_g6 

-.04 



DaysAbs_g6 

-.05 


(continued on next page) 
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Table A-3 
(continued) 


Outcome variable 

Intercept 

SD 

Level- 1 

Level- 1 

predictor variable 

Fixed effect 

Educational achievement (continued) 




LtcyScalScr_g6 

.14 

.71 

Male 

-.10 




FreeMeal 

-.03 




LtcyScalScrgS 

.64 




MathScalScr g5 

.20 




PunTot_g6 

-.02 




DaysAbs_g6 

-.02 

LtcyPrf_g6 



Male 

-.32 

(logistic model) 



FreeMeal 

-.15 




LtcyScalScrgS 

2.25 




MathScalScr g5 

.70 




PunTot_g6 

-.06 

MathScalScr_g6 

.20 

.69 

Male 

-.02 




Minority 

-.03 




HomEngl 

-.02 




FreeMeal 

-.03 




LtcyScalScrgS 

.21 




MathScalScrgS 

.64 




PunTot_g6 

-.03 




DaysAbs_g6 

-.04 

MathPrf g6 

• • • 

• • • 

LtcyScalScrgS 

.71 

(logistic model) 



MathScalScrgS 

2.28 




PunTot_g6 

-.15 




DaysAbs_g6 

-.13 
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Table A-4 

Prediction of Infractions, Punishments, Attendance, and Achievement in g8-gl2 
(Parsimonious models; p<.01) 


Outcome variable 

Intercept 

SD 

Level- 1 

Level- 1 

predictor variable 

Fixed 

effect 

Infractions 





InfrTot g9 

.24 

.31 

Male 

.03 




Minority 

.08 




InfrTot g8 

.50 




LitMath g8 

- .03 

InfrTot glO 

.22 

.24 

Minority 

.06 




PunTot_g9 

.42 




DaysAbs_g9 

.04 




LitMathWrtg g9 

- .04 

InfrTot gll 

.16 

.32 

Male 

.02 




Minority 

.05 




InfrTot glO 

.52 




PLAN_gI0 

- .04 

InfrTot gI2 

.30 

.18 

Male 

.05 




Minority 

.05 




PunTot gll 

.36 
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Table A-4 
(continued) 


Outcome variable 

Intercept 

SD 

Level- 1 

Level- 1 

predictor variable 

Fixed 

effect 

Punishments 





PunTotgS 

.06 

.96 

Male 

.01 




Minority 

.01 




FreeMeal 

.01 




InfrTot g8 

.98 

PunTot_g9 

.08 

.95 

Male 

.01 




Minority 

.01 




PunTot_g8 

.03 




LitMath g8 

- .02 




InfrTot g9 

.96 

PunTotglO 

.09 

.94 

PunTot_g9 

.03 




LitMathWrtg g9 

- .02 




InfrTot glO 

.97 

PunTotgll 

.07 

.94 

Male 

.01 




PunTot glO 

.03 




PLAN_gl0 

- .02 




InfrTot gll 

.97 

PunTot_gl2 

.11 

.87 

Male 

.02 




PLAN_gl0 

- .02 




PunTotgll 

.04 




InfrTot gI2 

.93 
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Table A-4 
(continued) 


Outcome variable 

Intercept 

SD 

Level- 1 

Level- 1 

predictor variable 

Fixed 

effect 

Attendance 





DaysAbs_g8 

.37 

.03 

Minority 

- .13 




HomEngl 

.05 




FreeMeal 

.14 




PunTot_g8 

.08 

DaysAbs_g9 

.31 

.36 

Male 

- .04 




FreeMeal 

.04 




DaysAbs_g8 

.58 




PunTot_g9 

.05 

DaysAbsglO 

.33 

.38 

Male 

- .04 




Minority 

- .04 




FreeMeal 

.05 




DaysAbs_g9 

.58 




PunTotglO 

.06 

DaysAbs gll 

.30 

.32 

Male 

- .07 




Minority 

- .05 




FreeMeal 

.06 




DaysAbs glO 

.56 




PLAN_glO 

- .02 




PunTotgll 

.07 

DaysAbs gl2 

.45 

.34 

Male 

- .04 




Minority 

- .03 




DaysAbs glO 

.48 




ELAgl 1 

- .04 




PunTot_gl2 

.10 
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Table A-4 
(continued) 


Outcome variable 

Intercept 

SD 

Level- 1 

Level- 1 

predictor variable 

Lixed 

effect 

Educational achievement 





LitMath g8 

21 

.14 

Male 

- .04 




Minority 

- .27 




LreeMeal 

- .12 




PunTotgS 

- .07 




DaysAbs_g8 

- .07 

LitMathWrtg g9 

.14 

.72 

Minority 

- .08 




LitMath g8 

.81 

PLAN_glO 

.09 

.71 

LitMthWrgt g9 

.84 




DaysAbsglO 

- .03 

ELAgl 1 

.17 

.60 

Male 

- .11 




Minority 

- .10 




LreeMeal 

- .06 




PunTot glO 

- .02 




PLAN_glO 

. 69 




DaysAbs gll 

- .04 

ACT_gll/12 

.09 

.79 

Male 

.09 




Minority 

- .04 




PLAN_glO 

.60 




DaysAbs gll 

-.02 




LLAgl 1 

.31 




PunTot_gl2 

- .02 




DaysAbs gl2 

- .03 

ACTCRB 

• • • 

• • • 

Male 

.48 

(logistic model) 



DaysAbs glO 

- .14 




PLAN glO 

2.00 




ELAgl 1 

.64 




DaysAbs gll 

- .18 
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